System for recording and playing audio signals provided on a photograph

ABSTRACT

A system for recoding and playing an audio signal provided on a photograph. The system includes a camera having a sensing system for sensing an image and recording an audio signal and a processor for generating coded data at least partially indicative of the audio signal and causing the photograph to be printed by printing the image and the coded data. The coded data is printed in substantially the same area of the photograph as the image. The system also includes a reader having a scanner for scanning the coded data, a processor for decoding the coded data to determine the audio signal and an output for playback of the audio signal.

RELATED APPLICATIONS

The present application is a Continuation of U.S. application Ser.No.10/309,068 filed on Dec. 4, 2002, which is a Continuation of U.S.application Ser. No. 09/693,078 filed on Oct. 20, 2000, now issued asU.S. Pat. No. 6,647,369.

FIELD OF THE INVENTION

The present invention relates to a camera system having an integralsound recording means for recording and outputting a permanent record ofsound associated with a photograph and recorded in infrared ink over thephotograph. Hence, the present invention relates to a photograph readerto decode sound and play sound.

The present invention also relates to a means for reproducing therecorded sound and discloses a photograph reader to decode sound andplay sound.

CO-PENDING APPLICATIONS

Various methods, systems and apparatus relating to the present inventionare disclosed in the following co-pending applications filed by theapplicant or assignee of the present invention simultaneously with thepresent application: US Patent Application Number 09/693,471 09/693,08309/693,134 09/693,226 09/693,317

The disclosures of these co-pending applications are incorporated hereinby reference.

Various methods, systems and apparatus relating to the present inventionare disclosed in the following co-pending application filed by theapplicant or assignee of the present invention on Jul. 10, 1998:

U.S. Ser. No. 09/113,060

U.S. Ser. No. 09/113,070

U.S. Ser. No. 09/112,741

U.S. Ser. No. 09/112,785

U.S. Ser. No. 09/113,107

The disclosures of this co-pending application are incorporated hereinby reference.

Various methods, systems and apparatus relating to the present inventionare disclosed in the following co-pending applications filed by theapplicant or assignee of the present invention on Jun. 30, 2000:

U.S. Ser. No. 09/608,308

U.S. Ser. No. 09/608,779

U.S. Ser. No. 09/607,987

U.S. Ser. No. 09/608/776

U.S. Ser. No. 09/607,250

U.S. Ser. No. 09/607,991

The disclosures of these co-pending applications is incorporated hereinby reference.

BACKGROUND OF THE INVENTION

The production of images by means of photographic techniques have beenwell known for a substantial period of time. Further, recently, digitalcameras have become increasingly popular where an image is captured bythe digital camera device and stored for later printing.

In applicant's applications U.S. Ser. Nos. 09/113,060, 09/113,070 andU.S. Ser. No. 09/113,222, a camera system has been proposed thatincludes an integral printer device for the printing out of sensedimages.

When using such devices and other image capture devices it will bedesirable to be able to suitably deal with audio and other environmentalinformation when taking a picture.

In applicant's application U.S. Ser. No. 09/113,107 one such device wasdisclosed in which audio data was recorded on the rear of a card orphotograph using black ink on a white background. A reader could readthe data and reproduce the signal through audio generator means when thecard or photograph was passed through a scanner means.

In U.S. Pat. No. 5,896,403 (Nagasaki et al.) a printing system isdisclosed which prints images and audio information on print media, forexample, a sheet, in dot codes which can be read by a pen device. Theinformation can be decoded notwithstanding scanning of the data beingirregular or random due to the organization of the dot codes. InNagasaki et al. system the scanning of data is low. In applicant'sprinting and scanning system as disclosed in U.S. Ser. No. 09/113,107the process is a pagewidth process rather than a line by line process asin Nagasaki et al.

SUMMARY OF THE INVENTION

The present invention relates to a camera system able to record audiowhen acquiring a sensed image in addition to being able to playback theaudio information when viewing a “photograph”, the audio and“photograph” being recorded on the same surface of a print media.

In accordance with a first aspect of the present invention, there isprovided a system for playing prerecorded audio encoded in a faulttolerant manner as an array of invisible ink dots printed on aphotograph in substantially the same area thereas comprising: aninfrared scanner means for scanning the printed invisible ink dots ofsaid prerecorded audio; a processor means interconnected to saidinfra-red scanner means for decoding said scanned audio encoding toproduce a corresponding audio signal; and audio emitter meansinterconnected to said processor means for emitting or playing saidcorresponding audio signal on demand. A further embodiment of thepresent invention provides a system for playing prerecorded audio

-   -   encoded in a fault tolerant manner as an array of invisible ink        dots printed on a card in    -   substantially the same area thereas comprising:        -   an infra-red scanner configured to scan the printed            infra-red ink dots of said prerecorded audio;        -   a decoder interconnected to said infra-red scanner and            configured to decode said scanned audio encoding and to            provide a corresponding output; and        -   an audio emitter interconnected to the decoder for emitting            audio corresponding to said output on demand.

The invisible ink may be an infrared (IR) absorbing ink with negligibleabsorption in the visible spectrum.

The encoding can include Reed-Solomon encoding of the prerecorded audioand comprises an array of IR (infrared) ink dots which are printed onthe photograph using a page width ink jet printhead. The array of dotsmay be high frequency modulated to aid scanning, for example using acheckerboard pattern.

The system can include a wand-like arm having a slot through which isinserted the photograph.

In accordance with a further aspect of the present invention, there isprovided a method of decoding a prerecorded audio signal printed as anarray of invisible ink dots in a fault tolerant encoded form on asurface of a photograph coincident with an image printed on said surfaceof said photograph, said method comprising the steps of:

-   -   (a) scanning the printed encoded form of said prerecorded audio        on said photograph in the infra-red;    -   (b) decoding said encoded audio signal; and    -   (c) playing said audio signal on an audio output device.

BRIEF DESCRIPTION OF THE DRAWINGS

Notwithstanding any other forms which may fall within the scope of thepresent invention, preferred forms of the invention will now bedescribed, by way of example only, with reference to the accompanyingdrawings in which:

FIG. 1 illustrates schematically the camera system constructed inaccordance to the preferred embodiment;

FIG. 2 illustrates schematically a printer mechanism for printing on theoutput media;

FIG. 3 illustrates a format of the output data on the photograph;

FIG. 4 illustrates an enlarged portion of the output media;

FIG. 5 illustrates a reader device utilized to read data from thephotograph; and

FIG. 6 illustrates the utilization of an apparatus of the preferredembodiment.

DESCRIPTION OF THE PREFERRED AND OTHER EMBODIMENTS

The preferred embodiment is preferably implemented through suitableprogramming of a hand held camera device such as that described in U.S.Ser. No. 09/113,060 and U.S. Ser. No. 09/113,070.

The aforementioned patent specifications disclose a camera system,hereinafter known as an “Artcam” type camera, wherein sensed images canbe directly printed out by an Artcam portable camera unit. Further, theaforementioned specifications disclose means and method for performingvarious manipulations on images captured by the camera sensing deviceleading to the production of various effects in any output image. Themanipulations are disclosed to be highly flexible in nature and can beimplemented through the insertion into the Artcam of cards havingencoded thereon various instructions for the manipulation of images, thecards being known as Artcards. The Artcam further has significantonboard processing power implemented by an Artcam Central Processor unit(ACP) which is interconnected to a memory device for the storage ofimport data and images.

In the preferred embodiment, the Artcam device is suitably modified soas to equip it with a microphone device and associated recordingtechnologies. When a picture is taken, the opportunity is provided torecord either the surrounding sound environment or a message associatedwith the image. The recorded audio is then printed on the photograph inan encoded format, the encoding preferably being of a highly resilientform. The recorded audio provides a permanent audio record associatedwith the photograph. Subsequently, a playback apparatus is provided forscanning the encoded audio and decoding this information.

Turning now to FIG. 1, there is illustrated, in schematic form thepreferred embodiment 1 which includes the arrangement as described inthe aforementioned patent specifications wherein an image 2 is sensedvia a CCD sensor 3 and forwarded to an Artcam central processor 4 whichincludes significant computational resources as described in theaforementioned patent specifications. The Artcam central processor 4 canstore the image in memory 5 which preferably comprises a high speedRAMBUS (Trade Mark) interfaced memory. The Artcam central processor 4 isalso responsible for controlling the operation of a printer device 6having a page width ink jet printhead 15 (see FIG. 2) for the printingout of full color photographs, eg. 7, so as to provide for instantimages on demand. The printhead 15 can print color images and has atleast (4) ink jet nozzles for printing at a resolution of 1600 dpi. Thefour inks include an infra-red ink for printing data, and cyan, magentaand yellow inks for printing a color image. Such a printhead, but havingthe capacity to print with six (6) inks, is disclosed in applicant'sapplications U.S. Ser. No. 09/608,308, U.S. Ser. No. 09/608,779, U.S.Ser. No. 09/607,987, U.S. Ser. No. 09/608,776, U.S. Ser. No. 09/607,250and U.S. Ser. No. 09/607,991 using a monolithic construction.

In the preferred embodiment, the camera arrangement 1 is also suppliedwith a sound chip 10 which interfaces via RAMBUS bus 11 to memory 5under the control of the ACP processor 4. The sound chip 10 can be of astandard or specialised form and can, for example, comprise a DSPprocessor that takes an analogue input 12 from a sound microphone 13.Alternatively, with increasing chip complexities (Moore's Law), thefunctionality of sound chip 10 can be incorporated onto the ACP chipwhich preferably comprises a leading edge CMOS type integrated incircuit chip. It will be readily evident that many other types ofarrangements can be provided which fall within the scope of the presentinvention. The sound chip 10 converts the analogue input 12 to acorresponding digital form and forwards it for storage in memory 5. Therecording process can be activated by means of the depressing of abutton (not shown) on the camera device, the button being under thecontrol of the ACP processor 4 otherwise it can be substantiallyautomatic when taking a photograph. The recorded data is stored in thememory 5.

Turning now to FIG. 2, the camera arrangement preferably includes aprinter device 6 which includes printhead 15. The printhead 15 isutilized to print an image on print media 17 and at the same time printaudio (or other) information on the same face of print output media 17using an IR ink. Similar arrangements but for printing information onthe back of an output photo image are described in U.S. Ser. No.09/112,741, the contents of which are hereby incorporated by referencemutatis mutandis. By using a single printhead a simpler apparatus isprovided than as therein disclosed. The applicant has disclosed aninfrared ink suitable for this purpose in Australian Provisional PatentsApplications PQ9412 and PQ9376 both filed on Aug. 14, 2000 andapplicant's applications PQ9509 filed on Aug. 18, 2000, and PQ9571, andPQ9561 both filed on Aug. 21, 2000.

Turning now to FIG. 3, there is illustrated an example of output printedon the photographic media 17. The media 17 includes a photographic image18 along with which is printed information 20 and data 22. The detail ofthe image 18 is not shown for ease of illustration. The information 20can include location, date and time data with the location data beingprovided by means of keyboard input or, alternatively, through theutilization of attached positioning systems such as GPS or the like. Theinformation 20 is presented in a viewable form. The data 22 is providedin an encoded invisible (but infra-red visible) form with the beginningand end thereof marked by target boundaries 23. The format of theencoding can be many and various, however, preferably the encoding isprovided in a highly fault tolerant manner so as to tolerate scratches,grime, writing, wear, rotation, fading etc. One form of suitabletechnology is the printing technology utilized in the construction of“Artcards” as described in the aforementioned patent specifications U.S.Ser. No. 09/113,060 or as disclosed in U.S. Ser. No. 09/112,070 and U.S.Ser. No. 09/112,785 or the co-pending applications Docket Nos. U.S. Ser.No. 09/693,471, U.S. Ser. No. 09/693,083 and U.S. Ser. No. 09/693,134filed concurrently herewith. The encoding format relies heavily uponutilisation of Reed-Solomon encoding of the data to provide for a highdegree of fault tolerance. A portion of the data 25 is shown inschematic form in FIG. 4 and the data comprises an array of IR dots asprinted by the printhead 15 of FIG. 2 which is additionally modulated bya high frequency “checkerboard” pattern 21 added to the data so as toassist in sensing of the encoded data. The data 22, in this particularinstance, can comprise sound data as well as image information 20. Thedisclosed methodologies as discussed in applicant's co-pendingapplications Nos. U.S. Ser. No. 09/693,471, U.S. Ser. No. 09/693,083 andU.S. Ser. No. 09/693,134 provide for the storage of about 6 Megabytes ofarbitrary data on a photograph having a size of 4″×6″ (102 mm.×152 mm.)for data printed at a resolution of 1600 dpi.

When it is desired to “play back” the recorded audio, the photograph 17is passed through a reader device 26 which includes pinch rollers forpinch rolling the photograph 17 past a linear CCD sensor device 27sensitive to infra-red, the photograph 17 being illuminated by aninfra-red source of complimentary characteristics. The sensor device 27can be a suitably adapted sensor device as described in theaforementioned patent specification or in applicant's application U.S.Ser. No. 09/693,317 specification. Suitable sensors are found in flatbed scanners or facsimile machines. The sensors of the CCD device areprovided with lensing and/or filtering arrangements to optimise theirsensitivity and coverage to that of the photograph's size and thespectral characteristics of the IR ink. An ink with minimal response inthe wavelength range 400-700 mm and maximal response in the wavelengthrange 700 mm-1200 mm has been found suitable as disclosed in AustralianProvisional Patent Applications PQ9412 and PQ9376 filed on Aug. 14, 2000and applicant's applications PQ9509 filed on Aug. 18, 2000, and PQ9571,and PQ9561 filed on Aug. 21, 2000.

Referring now to FIG. 6, there is illustrated in schematic form theoperation of the audio reader device 26 of FIG. 5. The linear CCD IRsensor 27 is interconnected to a second Artcam central processor 28which is suitably adapted to read and decode the data stored ininfra-red on the photograph. The decoded audio information is stored inmemory 32 for playback via a sound processing chip 33 on speaker 29. Thesound processing chip 33 can operate under the control of the ACPdecoder 28 which in turn operates under the control of various userinput controls 33 which can include volume controls, rewind, play andfast forward controls etc. Importantly, the CCD linear IR sensor 27 andthe ACP decoder 28 can implement the reading process as if theinformation were printed in an Artcard or an alternative Artcard formatas described in the aforementioned patent specifications U.S. Ser. No.09/113,070 and U.S. Ser. No. 09/112,785, or applicant's co-pendingapplications Nos. U.S. Ser. No. 09/693,471, U.S. Ser. No. 09/693,083 andU.S. Ser. No. 09/693,134.

It can be seen from the foregoing description of the preferredembodiment that there is provided a system for the automatic recordingof audio associated with an output image so as to provide an audiorecord associated with a photograph. There is also disclosed an audioreader system for reading an audio output recorded on a photograph withIR ink.

It would be appreciated by a person skilled in the art that numerousvariations and/or modifications any be made to the present invention asshown in the specific embodiment without departing from the spirit orscope of the invention as broadly described. For example, theutilisation of more complex audio recording and playback techniques suchas stereo and B-format techniques. The present embodiment is, therefore,to be considered in all respects to be illustrative and not restrictive.

1. A system for recoding and playing an audio signal provided on aphotograph, the system including: a camera including: a sensing systemfor sensing an image and recording an audio signal; and, a processorfor: generating coded data at least partially indicative of the audiosignal; causing the photograph to be printed by printing: the image; thecoded data, the coded data being printed in substantially the same areaas the image. a reader including: a scanner for scanning the coded data;a processor for decoding the coded data to determine the audio signal;and, an output for playback of the audio signal.
 2. A system accordingto claim 1, wherein the sensing system includes a CDD image sensor.
 3. Asystem according to claim 1, wherein the sensing system includes amicrophone coupled to a sound processor for digitising an analogue audiosignal generated by the microphone to thereby generate a digital audiosignal.
 4. A system according to claim 1, wherein the processor iscoupled to a memory for temporarily storing at least one of image dataand the audio signal prior to printing the photograph.
 5. A systemaccording to claim 1, wherein the camera includes a printer for printingthe photograph.
 6. A system according to claim 1, wherein the coded datais printed using infra-red responsive ink and the scanner is aninfra-red scanner.
 7. A system according to claim 1, wherein the readerincludes an arm having a slot through which said card is inserted.
 8. Asystem according to claim 7, wherein the scanner is located in the arm.9. A system according to claim 7, wherein the arm includes pinch rollerspositioned at either end of the slot, the pinch rollers being adapted toalign photographs with the scanner.
 10. A system according to claim 1,wherein the coded data is adapted to be scanned by a scanner extendingin a scan direction as the photograph moves passed the sensor in a feeddirection, the feed direction and the scan direction being substantiallyorthogonal.
 11. A system according to claim 10, wherein the coded dataincludes targets indicative of at least one of the beginning and end ofthe data.
 12. A system according to claim 11, wherein the targets aredisplaced from an array of dots representing the audio signalsubstantially in the feed direction.
 13. A system according to claim 1,wherein the coded data is printed as an array of substantially invisibledots.
 14. A system according to claim 1, wherein the coded data isredundantly encoded using Reed-Solomon encoding.
 15. A system accordingto claim 1, wherein the coded data is encoded using a high frequencycheckerboard modulation of an array of ink dots.
 16. A system accordingto claim 1, wherein the photograph includes information including atleast one of: photograph location; photograph date; and, photographtime.
 17. A method of decoding coded data provided on a photograph usinga reader, the coded data being at least partially indicative of an audiosignal and being provided on a photograph substrate in the same area asan image, the method including: scanning the image using a scanner;using coded data sensed by the scanner to determine an audio signal;and, playing the audio signal through an output.
 18. A method accordingto claim 17, wherein the method includes inserting the photographthrough a slot provided in an arm of the reader to thereby cause thecoded data to be scanned by a scanner in the arm of the reader.
 19. Amethod according to claim 17, wherein the coded data includes: an arrayof invisible dots indicative of the audio signal; and, targetsindicative of at least one of the beginning and end of the data.
 20. Amethod according to claim 19, wherein the method includes: detecting thetargets as the photograph is moved passed the scanner; and, using thedetected targets: sensing the array of dots; and, decoding the audiosignal.